Differentiable channel pruning guided via attention mechanism: a novel neural network pruning approach

Abstract Neural network pruning offers great prospects for facilitating the deployment of deep neural networks on computational resource limited devices. architecture search (NAS) provides an efficient way to automatically seek appropriate design compressed model. It is observed that, existing NAS-based methods, there usually a lack layer information when searching optimal architecture. In this paper, we propose new NAS approach, namely, differentiable channel method guided via attention mechanism (DCP-A), where adopted able provide guide optimization policy. The training process with Gumbel-softmax sampling, while parameters are optimized under two-stage procedure. block shortcut dedicatedly designed, which help prune not only its width but also depth. Extensive experiments performed verify applicability and superiority proposed method. Detailed analysis visualization pruned model shows that our DCP-A learns explainable policies.